High Dimensional Separable Representations for Statistical Estimation and Controlled Sensing

High Dimensional Separable Representations for Statistical Estimation and Controlled Sensing by Theodoros Tsiligkaridis Chair: Alfred O. Hero III Separable approximations are effective dimensionality reduction techniques for high dimensional data. The statistical estimation performance of separable models is largely unexplored in high dimensions and model mismatch errors need to be accurately controlled. The need for performance bounds associated with statistical estimators in sample starved settings has been a topic of great interest in the field of signal processing and high-dimensional statistics. Many signal processing methods, including classical filtering, prediction and detection, are intimately linked to the data covariance. In multiple modality and spatio-temporal signal processing, separable models for the underlying covariance may be exploited for dimensionality reduction, improved estimation accuracy and reduction in computational complexity. In controlled sensing (or inference), estimation performance can be greatly optimized at the expense of query design (or control). Query-based multisensor controlled sensing systems used for target localization consist of a set of sensors (possibly heterogeneous and of different modality) that collaborate (through a fusion center or by local information sharing) to estimate the location of a target. In the centralized setting, at each time instant, a fusion center designs queries for the sensors on the presence of the target in a given region and noisy responses are obtained. For a large number of sensors and/or high-dimensional targets, separable representations of the query policies can be exploited to maintain tractability. For very large sensor networks, decentralized estimation methods are of primary interest and local message-passing techniques can be exploited to increase flexibility, robustness and scalability. Motivated by this fundamental set of high dimensional problems, the thesis makes contributions in the following areas: (1) performance bounds for high dimensional estimation for structured Kronecker product covariance matrices, (2) optimal query design for a centralized collaborative controlled sensing system used for target localization, and (3) global convergence theory of decentralized controlled sensing for target localization. A rich class of covariance models widely applicable to spatio-temporal settings are sums of Kronecker products (KP). For the special case of a single KP model with optional sparsity in the factors, a block-coordinate descent method used to solve the penalized MLE problem is proven to achieve a tight global MSE convergence rate in high dimensions. More generally, under a convex optimization framework, high dimensional MSE convergence rates are derived that show a fundamental tradeoff between estimation error and the approximation error induced by the dimensionality reduction on the space of covariance matrices in terms of KP’s. The results improve upon the current state-of-the-art methods. Under the minimum entropy criterion, the optimality conditions for the joint policy for control of a centralized collaborative system of sensors for target search are